Minimax lower bounds
نویسنده
چکیده
Now that we have a good handle on the performance of ERM and its variants, it is time to ask whether we can do better. For example, consider binary classification: we observe n i.i.d. training samples from an unknown joint distribution P on X× {0,1}, where X is some feature space, and for a fixed class F of candidate classifiers f :X→ {0,1} we let f̂n be the ERM solution f̂n = argmin f ∈F 1 n n ∑ i=1 1{ f (Xi ) 6=Yi }. (1)
منابع مشابه
Minimax Lower Bounds for Nonparametric Deconvolution Report 98{25
Suppose we have i.i.d. observations with a distribution equal to the convolution of an unknown distribution function F and a known distribution function K. We derive local minimax lower bounds for the problem of estimating F , its density f and its derivatives at a xed point x 0. Contrary to a previous local minimax bound in Van Es (1998) only smooth perturbations are considered. The local boun...
متن کاملMinimax Lower Bounds
Minimax Lower Bounds Adityanand Guntuboyina 2011 This thesis deals with lower bounds for the minimax risk in general decision-theoretic problems. Such bounds are useful for assessing the quality of decision rules. After providing a unified treatment of existing techniques, we prove new lower bounds which involve f -divergences, a general class of dissimilarity measures between probability measu...
متن کاملLower bounds for volatility estimation in microstructure noise models∗
In this paper minimax lower bounds are derived for the estimation of the instantaneous volatility in three related high-frequency statistical models. These bounds are based on new upper bounds for the Kullback-Leibler divergence between two multivariate normal random variables along with a spectral analysis of the processes. A comparison with known upper bounds shows that these lower bounds are...
متن کاملLocalized Upper and Lower Bounds for Some Estimation Problems
We derive upper and lower bounds for some statistical estimation problems. The upper bounds are established for the Gibbs algorithm. The lower bounds, applicable for all statistical estimators, match the obtained upper bounds for various problems. Moreover, our framework can be regarded as a natural generalization of the standard minimax framework, in that we allow the performance of the estima...
متن کاملOptimal Non-Asymptotic Lower Bound on the Minimax Regret of Learning with Expert Advice
We prove non-asymptotic lower bounds on the expectation of the maximum of d independent Gaussian variables and the expectation of the maximum of d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (asymptotically optimal) non-asymptotic lower bound on the minimax regret of onlin...
متن کاملInformation-theoretic lower bounds for distributed statistical estimation with communication constraints
We establish lower bounds on minimax risks for distributed statistical estimation under a communication budget. Such lower bounds reveal the minimum amount of communication required by any procedure to achieve the centralized minimax-optimal rates for statistical estimation. We study two classes of protocols: one in which machines send messages independently, and a second allowing for interacti...
متن کامل